A validation study demonstrating portable motion capture cameras accurately characterize gait metrics when compared to a pressure-sensitive walkway

Digital quantification of gait can be used to measure aging- and disease-related decline in mobility. Gait performance also predicts prognosis, disease progression, and response to therapies. Most gait analysis systems require large amounts of space, resources, and expertise to implement and are not widely accessible. Thus, there is a need for a portable system that accurately characterizes gait. Here, depth video from two portable cameras accurately reconstructed gait metrics comparable to those reported by a pressure-sensitive walkway. 392 research participants walked across a four-meter pressure-sensitive walkway while depth video was recorded. Gait speed, cadence, and step and stride durations and lengths strongly correlated (r > 0.9) between modalities, with root-mean-squared-errors (RMSE) of 0.04 m/s, 2.3 steps/min, 0.03 s, and 0.05–0.08 m for speed, cadence, step/stride duration, and step/stride length, respectively. Step, stance, and double support durations (gait cycle percentage) significantly correlated (r > 0.6) between modalities, with 5% RMSE for step and stance and 10% RMSE for double support. In an exploratory analysis, gait speed from both modalities significantly related to healthy, mild, moderate, or severe categorizations of Charleson Comorbidity Indices (ANOVA, Tukey’s HSD, p < 0.0125). These findings demonstrate the viability of using depth video to expand access to quantitative gait assessments.

The US population is aging rapidly, and soon the number of adults aged 65 and older is anticipated to outnumber children for the first time in US history 1 .With an aging population, the incidence of cognitive disorders and neurologic diseases is also anticipated to increase [2][3][4] .Neurological and physiological changes related to aging have numerous effects on one's health, including decline in mobility 5 .Gait changes are a significant concern for older adults because they lead to falls, disability, and loss of independence 6,7 .Gait abnormalities can also be critical for detecting and tracking the progression of neurologic diseases 8 .Abnormalities in gait have been correlated with a higher future risk of developing dementia 9,10 .Gait speed has been shown to have a strong correlation with survival amongst older adults, where a 0.1 m/s change has been associated with increased mortality 11 .Thus, quantified gait assessments are a critical first step for identifying potential gait abnormalities and subsequently their implications on an individual's health span.
There are numerous approaches for quantifying gait patterns using devices such as marker-based motion capture systems, walkways, treadmills, and wearables [12][13][14][15][16] .Instrumented walkways provide detailed spatiotemporal information about the timing of each foot fall, the step and stride lengths, asymmetry, or variability between strides, as well as numerous other gait metrics [17][18][19] .Camera-based motion capture systems are another established technology for characterizing the kinematics of a patient's gait 20,21 .Traditionally, motion capture systems

Results
Gait assessments from 392 consecutive patients enrolled in the Mayo Clinic Study of Aging (MCSA) were analyzed as part of this study.The MCSA is a population-based cohort, patients in this study were recruited from Olmsted County Minnesota using a consecutive sampling approach.The median age of the participants at the time of the walking assessment was 73 years (51 and 89 years for the 5th and 95th percentiles, respectively), with equal numbers of male and female participants (196 each).Patients walked across a pressure-sensitive walkway and were simultaneously recorded using depth video (see Methods).Gait metrics were derived from both depth video data and the walkway.These metrics are referred to as "depth video" and "walkway" metrics, respectively.
Figure 1 depicts a schematic of the room in which participants walked across the walkway, as well as the center of mass (com) and toe markers for the left and right limbs for one participant.The z-direction was re-oriented Figure 1.Schematic of the camera and walkway layout and the step detection algorithm for the left and right limbs (left and right panels, respectively).(a) Cartoon depiction of the placement of the walkway and cameras in the room.Participants entered the room and walked along the z-axis dimension across the walkway.The x-axis pointed to the left wall and the y-axis pointed up towards the ceiling.Camera #1 was mounted on the near wall and camera #2 was mounted on the far wall based on where physical mounts could be placed in the room.The width of the walkway was 1.2 m across, and dashed lines extend to the near wall to show how this is represented in the three-dimensional perspective of the schematic.The far wall and right wall are labeled in italics in the schematic, the near wall and left wall were not for simplicity of the drawing.(b) Position of the center of mass (com) of the body moving across the walkway in the z-direction.(c) Position of the toe marker moving across the walkway in the z-direction.(d) Difference between the toe marker and the com marker in the z-direction.Orange circles indicate peaks when the toe is maximally in front of the center of mass.Green circles indicate when the toe is maximally behind the center of mass.
to align with the forward movement of the person.Steps were identified based on the valid sequence of up and down events for the left and right limbs (see Methods for more details).The median number of strides detected by the walkway was 15 (12, 22 strides for the 5th and 95th percentiles, respectively) compared with 12 (8, 21 strides) from the depth video.Figure 2 depicts the cadence measured from the depth video and the walkway.There was a strong correlation between the two modalities (r = 0.98) with a root-mean-square error (RMSE) of 2.3 steps/min between modalities.The Bland-Altman plot 47 shows there was a bias of + 0.6 steps/min (− 3.8, 5.1 steps/min, lower and upper limits, respectively), with 96.2% of the data within the limits of agreement (LOA), consistent with what would be expected 48 .

Accuracy of the distance and temporal gait metrics calculated from depth information
Temporal-and distance-based measures of gait have been shown to be strong indicators of aging in older adults 11 .Here, we found strong correlations for distance and temporal gait metrics calculated from the depth video and the walkway (Table 1, Fig. 3).Left and right limb stride length metrics had correlation coefficients of 0.98 for each limb with RMSE of 0.05 m for left strides and 0.04 m for right strides.Based on the Bland-Altman plots in Fig. 3a, there was a mean difference of − 0.019 m (− 0.11, 0.07 m lower and upper LOAs, 97.2% of data within LOAs) between the depth video and the walkway for left stride lengths (negative value indicates the stride length derived from the depth video was less than the walkway measurement).For the right limb (Fig. 3b), the mean difference was − 0.005 m (− 0.09, 0.08 m lower and upper LOAs, 96.7% of data within LOAs).For the step length metrics (Fig. 3c and 3d), the left limb had a correlation of 0.92 and a RMSE of 0.05 m, and the right limb had a correlation of 0.91 and a RMSE of 0.08 m between the depth video and walkway.Bland-Altman plots again revealed a mean difference of − 0.02 m (0.06 m, − 0.10 m lower and upper LOAs, 94.9% of data within LOAs) for the left limb and 0.06 m (− 0.03, 0.15 m lower and upper LOAs, 96.7% of data within LOAs) for the right limb.
When analyzing the durations of the stride and steps, a similarly strong correlation and consistency was observed between the depth video and walkway.For the stride durations, both the left and right limb metrics had correlations of 0.98 and RMSEs of 0.03 s for each limb (Fig. 3c).The left limb stride duration had a mean difference of − 0.003 s (− 0.05, 0.05 s lower and upper LOAs, 95.9% of data within LOAs).Right limb stride durations had a mean bias of − 0.005 s (− 0.05, 0.04 s lower and upper LOAs, 95.7% of data within LOAs).Similarly for the step durations (Fig. 3d), the left limb had a correlation of 0.90 and the right limb had a correlation of 0.92, with 0.03 s RMSE for each limb.The Bland-Altman plots indicated there was a − 0.012 s mean difference (− 0.07, 0.04 s, lower and upper LOAs, 93.9% of data within LOAs) for the left limb and a + 0.009 s bias (− 0.09, 0.08 s, lower and upper LOAs, 93.4% of data within LOAs) for the right limb.
Measures of gait speed, calculated by dividing the stride length by the stride duration, also showed strong correlations between the depth video and the walkway (r = 0.98 both left and right limbs) with RMSEs of 0.04 m/s for each limb.The Bland-Altman analyses revealed a mean difference of − 0.003 m/s (− 0.09, 0.08 m/s lower and upper LOAs, 96.7% of data within LOAs) for the gait speed calculated from left strides and − 0.001 m/s (− 0.09, 0.08 m/s lower and upper LOAs, 97.2% of data with LOAs) for the gait speed calculated from right strides.

Accuracy of identifying different gait cycle phases depth information
Next, we examined how well we could determine the duration of different phases of the step cycle (compared to those detected by the pressure-sensitive walkway).Swing duration, stance duration, and double support duration were each reported as a percentage of gait cycle (Table 2).When comparing the depth video and walkway metrics (Fig. 4a and 4b), the swing duration had a correlation of 0.62 for the left limb and 0.60 for the right limb.The left limb swing duration had a 6% RMSE and the right limb swing duration had a 5% RMSE.Bland-Altman plots revealed that both metrics had a mean difference of 5 percentage points (0.9, 9.9% lower and upper LOAs for left, 92.9% of data within LOAs; 0.7, 9.3% lower and upper LOAs for right, 95.9% of data within LOAs).For the stance duration (Fig. 4c and 4d), the correlations were again 0.62 for the left limb and 0.60 for the right limb, and 6% and 5% RMSE for the left and right limb stance duration, respectively.Based on the Bland-Altman plots, there was a mean difference of − 5 percentage points for both metrics (− 9.9, − 0.9% lower and upper LOAs for left, 92.9% of data within LOAs, − 9.3, − 0.7 lower and upper LOAs for right, 95.9% of data within LOAs).For the double stance duration, the correlation was 0.69, 11% RMSE, and mean difference of − 10 percentage points for each limb (− 17.2, − 2.9% lower and upper LOAs for left, 94.9% of data within LOAs; − 17.4, − 3.2% lower and upper LOAs, 95.2% of data within LOAs).Overall, the symmetry with which the depth video overestimated the stance duration and underestimated the swing phase could be due to differences with how the toe down and toe up events were detected between modalities.

Gait Speed significantly relates to comorbidity scores similarly for both modalities
Several studies have identified that reduced gait speed is a significant predictor of increased risk of mortality in older adults 11 .As an exploratory analysis, we selected a subset of 336 participants for which we had Charleson Comorbidity Index (CCI) scores 49 .Participants were categorized into healthy, mild, moderate, or severe morbidity risk if their CCI scores were 0, 1-2, 3-4, and ≥ 5, respectively.Of the 336 participants, 58 were healthy (17.26%), 100 were mild (29.76%), 73 were moderate (21.73%), and 105 were severe (31.25%).There was a significant effect of gait speed for the different CCI categories (Fig. 5, ANOVA, p < 0.025).Based on the Tukey's HSD post-hoc tests, the gait velocity for individuals categorized as healthy or mild was greater than those categorized as moderate (p < 0.01) or severe (p < 1e−5) individuals.However, there was no significant difference between healthy and mild, or moderate and severe.The same pattern was detected from gait velocities for the right limb, with the same statistical values (ANOVA, p < 0.025, Tukey's HSD p < 0.01 for healthy or mild vs moderate and p < 1e−5 for healthy or mild vs. severe).Importantly, the outcomes were identical using either the depth video or walkway to derive the gait metrics, demonstrating depth video can extract gait metrics with clinically meaningful information.

Discussion
We have successfully demonstrated that depth information collected from two portable cameras can accurately reconstruct gait metrics on par with a gold standard pressure-sensitive walkway.There was a strong correlation between measurements made by the two modalities for distance measures (e.g., step length and step duration) which are important clinical markers for assessing gait abnormalities.High degree of agreement with small errors demonstrated validity while small RMSE values demonstrated precision of the depth video compared to the pressure-sensitive walkway.The agreement between the different phases of the gait cycle was fairly accurate but not as strong as the distance measures.This could be due to differences in when the walkway detects the foot lifting and pressing down on the ground compared to what was derived from the depth video.The depth video detected these events based on when the toe marker was maximally in front or behind the body center-of-mass.
The toe marker may begin to make contact or release from the walkway before these points, which could explain the difference.However, the kinematic video information does not have the same kinetic precision as the walkway and thus a more refined foot detection algorithm could resolve these differences (e.g., by using machine learning approaches to identify the timing of toe down and toe up against the two modalities).Importantly, the depth video metrics were consistent with the walkway for each of the gait metrics calculated.The statistical pattern relating gait velocity to CCI scores (Fig. 5) demonstrates that the metrics derived are similar between modalities but also relate to clinically meaningful measures.Overall, these findings demonstrate the utility of such a system for clinical use.The population assessed in this study was relatively normal in relation to known movement disorders, so future work would be needed to validate the technology on patients with more pronounced gait abnormalities such as patients with Parkinson's disease or normal pressure hydrocephalus.However, the findings here are a promising step toward remotely tracking aging or neurologic disease progression using gait metrics 8 .The proposed system only uses depth (point-cloud) data to track the joint positions.This is advantageous from a patient privacy perspective because although gait is an identifying characteristic, omitting the use of recognizable facial and body features at least removes further identifying information to alleviate some privacy concerns 44 .Depth cameras have been shown to be limited in their capture rate and the size of the capture space which can affect the accuracy of detecting individual joint positions during walking 50,51 .Incorporating color video to supplement the point-cloud data and potentially improve marker tracking could be helpful, however at the tradeoff of requiring more identifying features (e.g., facial features) to be recorded.Another potential limitation of the proposed setup is human intervention is required to process the depth video and track the skeletal markers.Although a fully automated system would be more efficient, this human intervention step provides a quality control check to ensure the tracking software accurately detected the person in view.Additionally, in our study the tracking step was brief to perform given the nature of the movements being tracked.
Overall, the agreement of these systems suggests a portable system could be a viable solution for performing initial gait assessments for patients at medical centers where implementing a more resource intensive gait lab might not be possible.This approach should not be considered as a replacement for pressure-sensitive walkways or motion capture systems.The portability of this system also expands access for performing longitudinal assessments of gait which is crucial for early detection of potential gait abnormalities and tracking gait-based biomarkers related to the progression of different neurologic diseases.In this cohort of relatively healthy individuals, minor differences were observed between the gait metrics reported by the pressure-sensitive walkway and the depth video, such as differences in footfall detection or offsets swing and stance phase durations.However, these differences were within acceptable limits when assessing the accuracy and precision of gait metrics between modalities.Before implementing such a system for clinical applications, further research is needed to assess the system's accuracy and precision in a diverse population of patients with different neurologic diseases or movement disorders to ensure the performance is within acceptable limits.Clinical use would require further validation and assessment of accuracy and precision in light of clinically meaningful change and disease heterogeneity.

Participants
We conducted a validation study of 392 participants as part of the Mayo Clinic Study of Aging (MCSA) using a consecutive sampling approach.Part of their assessments involves walking across a 5 m pressure-sensitive walkway (ZenoTM Walkway, ProtoKinetics LLC, Havertown, Pennsylvania).Participants consented to also have depth information be recorded as part of this study.All participants provided informed consent prior to their inclusion in the study, and all procedures and protocols of this study conformed to the Declaration of Helsinki and were approved by the Mayo Clinic Institutional Review Board.

Walking assessment
Enrolled participants performed a walking assessment in which they walked over a 5 m pressure-sensitive walkway two times at their own pace.A schematic of the room is depicted in Fig. 1a.Study coordinators instructed the participants to stand at the entrance of the assessment room, walk straight across the assessment room, turn around, and walk back to their starting point at their normal speed.Participants began walking and turned around while not on the walkway such that the averaged gait metrics from each step more closely corresponded to their steady state walking.Two Azure HD Kinect (Microsoft, Washington, USA) cameras were placed in the room which were synchronized to a laptop using the iPi Recorder software (iPi Soft LLC, Lamarjean Group, Inc, Poway, CA) to record depth information related to their walking.The cameras nominally recorded at 30 frames per second (FPS), however for some sessions the frame rate dropped due to issues synchronizing the two cameras.For these sessions, we used the average frame rate reported by the iPi Recorder when analyzing the data.
Although the cameras can record both RGB and depth information, only depth information was recorded here to prevent recording personally identifiable information such as facial features.Both cameras were calibrated using a small flashlight as described by the manufacturer's instructions.In short, this involved moving a small flashlight throughout the capture space while recording a calibration file using the iPi Recorder software.This calibration file was then used for analyzing the subsequent patient recordings.The positions of the cameras were placed based on the physical configurations of the room as well as the manufacturer's recommendations for the capture space dimensions.As depicted in Fig. 1a, the walkway was 0.9 m from the far wall and 2.2 m from the near wall.Two Azure HD Kinect cameras were mounted 0.3 m above the ground on the near wall (camera #1) and the far wall (camera #2).Based on physical restrictions in the room, the walkway was not completely centered, with the walkway being 1.0 m from the left wall and 0.8 m from the right wall.

Depth video data preprocessing and extracting gait metrics
The depth recordings were processed using the iPi Mocap Studio 4 Software (iPi Soft LLC).A skeleton was fit to the depth point cloud data to track the person walking during the assessment.The turn was excluded from analyses because only one camera had coverage which was insufficient for accurately tracking.The skeleton trajectory was smoothed using the iPiSoft jitter removal tool to remove potential outlier datapoints.Once the depth data were processed, the joint coordinates were exported using the biomech add-in to obtain absolute positions relative to the global ground in the tracking software.The default skeleton available in the iPi Motion Studio provided the following marker coordinates: center of mass, center of mass projection to ground, hip, lower spine, middle spine, chest, neck, head, eye, effector head, clavicle (Left/Right), shoulder (L/R), forearm (L/R), hand (L/R), thigh (L/R), shin (L/R), foot (L/R), toe (L/R), effector toe (L/R).The cartesian values of the joint coordinates were outputted in tabular format text files which were processed in python for subsequent analysis.These text files contained header information to select specific joint marker information, such as toe or com coordinates, for subsequent stride detection.Placement of the cameras did not capture when participants began walking or when they turned around to be consistent with the gait information collected by the walkway.The center of mass ( com ) and left and right toe markers were used for detecting strides during the session.Any missing frames were interpolated using piecewise cubic hermite interpolating polynomial (pchip) algorithm 52 .Next, we confirmed the person was walking straight and not turning in the middle of the session.We used the com x and com z positions and subtracted the first frame to ensure the person was walking forward in the com z direction.Turns were identified based on positive or negative com z peaks (scipy find_peaks function, promi- nence = 1).Time steps between these turn events were excluded from additional analysis.The com z and toe z positions were used to detect strides.Strides were defined from toe-down to toe-down ( t down[n+1] − t down[n] ) of the ipsilateral limb.First, the toe z position was subtracted by the com z position to find the relative change in the foot from the body.Toe down was defined as the positive peak of toe z from the com z (find_peaks, prominence 0.5 standard deviations of the overall toe z ).Similarly, toe up was defined as the negative peak of toe z from the com z (in the backward direction).These events were then labeled as R down , R up , L down , and L up depending on which toe z marker was being analyzed.This resulted in a sequence of events that was then used to detect "valid strides."We defined a valid right stride as sequence of R down − L up − L down − R up − R down and left stride as L down − R up − R down − L up − L down .These events also defined the gait cycle for which subsequent metrics were calculated.
Using the detected strides, we calculated the following gait metrics similar to those described in

Exploratory analysis comparing Charleson Comorbidity Index (CCI) to gait speed
The Charleson Comorbidity Index (CCI) scores provide a morbidity score reflecting one's mortality risk 49 .To calculate the score, 19 different medical conditions are assessed in addition to including a risk factor for the age of the patient.For a subset of 336 patients, CCI scores were available which were then divided into 4 categories similar to those reported by Huang et al. 54 : healthy/no-comorbidity (CCI = 0), mild (CCI = 1 or 2), moderate (CCI = 3 or 4), severe (CCI ≥ 5).As an exploratory analysis, we compared the gait speed metric from the left and right limbs separately to determine how changes in gait speed related to increased risk of mortality based on the CCI categories.Additionally, the distributions were qualitatively compared when using gait speed derived from the depth video and reported by the walkway to examine the efficacy of using depth video metrics for clinical assessments.The seaborn python package 55 was used to create the violin plots comparing groups, and the statsmodels annotations package 56 was used to add significance bars for the post-hoc comparisons.

Statistical analyses
Each of the gait metrics was compared between the depth video and pressure to those reported by the pressuresensitive walkway.Pearson correlations and Bland-Altman plots were calculated to determine how well the values aligned for each metric.Bland-Altman results were presented as the bias of the difference between the two modalities (walkway, depth video) along with the lower and upper limits of agreement (LOAs, 1.96 × standard deviation) 47,57 .For the exploratory analysis, the Statsmodels python library 58 was used to perform ANOVA and Tukey's HSD post-hoc tests.The significance level for the post-hoc tests was Bonferroni corrected for testing both left and right limb gait metrics for both modalities (alpha = 0.05/(2 limbs × 2 modalities) = 0.0125).

Figure 2 .
Figure 2. Correlation plot and Bland-Altman plot comparing cadence metrics calculated from the depth video and reported by the walkway.(a) Correlation plot comparing cadence for the walkway (y-axis) vs the depth video (x-axis).Black dashed line indicates y = x.Correlation, p value, and RMSE annotated in the plot.(b) Bland-Altman plot indicating the difference between the depth video and the walkway vs the average value of the two modalities.Black dashed lines indicate the lower and upper LOAs, solid line indicates mean difference.Values are depicted to the right of the plot.

2 Figure 3 .
Figure 3. Correlation and Bland-Altman plots of distance gait metrics comparing both the left and right limbs.(a) Left stride length, (b) right stride length, (c) left step length, and (d) right step length.For each metric, the left-most plot depicts the Pearson correlation (r), p value, and RMSE value, and the right-most plot depicts the Bland-Altman plot with mean difference and LOAs.Conventions similar to Fig. 2.

Figure 4 .
Figure 4. Correlations and Bland-Altman plots for the phases of the gait cycle.(a) Left limb swing duration, (b) Right Limb swing duration, (c) Left limb stance duration, and d) Right limb stance duration.Each metric is represented as a percentage of the gait cycle.For each metric, the left-most plot depicts the Pearson correlation (r), p value, and RMSE value, and the right-most plot depicts the Bland-Altman plot with mean difference and LOAs.Conventions similar to Fig. 2.

Figure 5 .
Figure 5.Comparison of left limb gait speed with discretized CCI scores.CCI scores are categorized into Healthy (blue), Mild (dark orange), Moderate (green), and Severe (light purple).Significance bars are displayed at the top with p-values after performing Tukey's HSD post-hoc tests between groups.Metrics derived from the depth videos shown on the left, reported by the walkway shown on the right.

Table 1 .
Precision, accuracy, and correlation of temporal and distance metrics.

Table 2 .
Precision, Accuracy, and Correlation of Gait Cycle Phases.